[PHP] - Lowering script memory usage in a "big" file creation

Posted by Riccardo on Stack Overflow See other posts from Stack Overflow or by Riccardo
Published on 2010-06-15T08:32:09Z Indexed on 2010/06/15 8:32 UTC
Read the original article Hit count: 228

Filed under:

Hi there people, it looks like I'm facing a typical memory outage problem when using a PHP script.

The script, originally developed by another person, serves as an XML sitemap creator, and on large websites uses quite a lot of memory.

I thought that the problem was related due to an algorithm holding data in memory until the job was done, but digging into the code I have discovered that the script works in this way:

  • open file in output (will contain XML sitemap entries)
  • in the loop: ---- for each entry to be added in sitemap, do fwrite
  • close file
  • end

Although there are no huge arrays or variables being kept in memory, this technique uses a lot of memory.

I thought that maybe PHP was buffering under the hood the fwrites and "flushing" data at the end of the script, so I have modified the code to close and open the file every Nth record, but the memory usage is still the same.... I'm debugging the script on my computer and watching memory usage: while script execution runs, memory allocation grows.

Is there a particular technique to instruct PHP to free unsed memory, to force flushing buffers if any?

Thanks

© Stack Overflow or respective owner

Related posts about php